Mitigation strategies for lane marking misdetection

ABSTRACT

A vehicle, and a system and method of navigating the vehicle. The system includes a camera and a processor. The camera obtains an image of a road upon which the vehicle is moving. The processor is configured to extract a feature of the road from the image, perform a lane detection algorithm to detect a set of lane markers in the road using the image and the feature, and move the vehicle along the road by tracking the set of lane markers.

INTRODUCTION

The subject disclosure relates to lane detection in autonomous vehiclesand, in particular, to a system and method for detecting lane markingsin harsh or unfavorable environmental conditions.

Autonomous vehicles detect lane markings in a roadway in order tonavigate the roadway without diverging into other lanes. In goodenvironmental conditions, lane detection can be performed by taking animage of the roadway using a camera and identifying the lane markings inthe roadway using suitable algorithms. However, deterioratedenvironmental conditions can make it difficult to detect lane markingswithin the image using this method. For example, night or darkconditions yields dim images, and snow or heavy rain can obscure thelane markings on the roadway. Accordingly, it is desirable to provide amethod for detecting lane markings in a roadway in unfavorableenvironmental conditions.

SUMMARY

In one exemplary embodiment, a method of navigating a vehicle isdisclosed. An image is obtained at a camera on the vehicle of a roadupon which the vehicle is moving. A feature of the road is extractedfrom the image. A lane detection algorithm is performed to detect a setof lane markers in the road using the image and the feature. The vehicleis moved along the road by tracking the set of lane markers.

In addition to one or more of the features described herein, the featurefurther includes a road segmentation information. The feature furtherincludes a trajectory of a road agent on the road. The method furtherincludes displaying at a monitor the set of lane markers to a driver ofthe vehicle. The method further includes at least one of sending theimage to a lane detection module and sending the image to both the lanedetection module and an information extraction module. The methodfurther includes obtaining a first set of marker points by sending theimage to the lane detection module and a second set of marker points bysending the image to both the lane detection module and the informationextraction module and selecting a final set of marker points from thefirst set of marker points and the second set of marker points. Themethod further includes determining a first confidence value for thefirst set of marker points and a second confidence value for the secondset of marker points and selecting the final set of marker points usingthe first confidence value and the second confidence value.

In another exemplary embodiment, a system for navigating a vehicle isdisclosed. The system includes a camera and a processor. The cameraobtains an image of a road upon which the vehicle is moving. Theprocessor is configured to extract a feature of the road from the image,perform a lane detection algorithm to detect a set of lane markers inthe road using the image and the feature, and move the vehicle along theroad by tracking the set of lane markers.

In addition to one or more of the features described herein, the featurefurther includes a road segmentation information. The feature furtherincludes a trajectory of a road agent on the road. The processor isfurther configured to display the set of lane markers to a driver of thevehicle at a monitor. The processor is further configured to perform atleast one of send the image to a lane detection module and send theimage to both the lane detection module and an information extractionmodule. The processor is further configured to obtain a first set ofmarker points by sending the image to the lane detection module and asecond set of marker points by sending the image to both the lanedetection module and the information extraction module and select afinal set of marker points from the first set of marker points and thesecond set of marker points. The processor is further configured todetermine a first confidence value for the first set of marker pointsand a second confidence value for the second set of marker points andselect the final set of marker points using the first confidence valueand the second confidence value.

In another exemplary embodiment, a vehicle is disclosed. The vehicleincludes a camera and a processor. The camera obtains an image of a roadupon which the vehicle is moving. The processor is configured to extracta feature of the road from the image, perform a lane detection algorithmto detect a set of lane markers in the road using the image and thefeature, and move the vehicle along the road by tracking the set of lanemarkers.

In addition to one or more of the features described herein, the featurefurther includes at least one of a road segmentation information and atrajectory of a road agent on the road. The processor is furtherconfigured to display the set of lane markers to a driver of the vehicleat a monitor. The processor is further configured to perform at leastone of sending the image to a lane detection module and sending theimage to both the lane detection module and an information extractionmodule. The processor is further configured to obtain a first set ofmarker points by sending the image to the lane detection module and asecond set of marker points by sending the image to both the lanedetection module and the information extraction module and select afinal set of marker points from the first set of marker points and thesecond set of marker points. The processor is further configured todetermine a first confidence value for the first set of marker pointsand a second confidence value for the second set of marker points andselect the final set of marker points using the first confidence valueand the second confidence value.

The above features and advantages, and other features and advantages ofthe disclosure are readily apparent from the following detaileddescription when taken in connection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Other features, advantages and details appear, by way of example only,in the following detailed description, the detailed descriptionreferring to the drawings in which:

FIG. 1 shows a vehicle in accordance with an exemplary embodiment;

FIG. 2 shows a schematic diagram of a lane detection system foridentifying lane markers under several conditions;

FIG. 3 shows a schematic representation of the components of the lanedetection system operable at the processor of the vehicle to detect lanemarkings under difficult environmental conditions;

FIG. 4 shows a schematic diagram illustrating a first method fordetecting lane markings;

FIG. 5 shows a schematic diagram illustrating a second method fordetecting lane markings;

FIG. 6 shows a schematic diagram illustrating a third method by whichthe lane detection system detects lane markings;

FIG. 7 shows a schematic diagram illustrating a fourth method by whichthe lane detection system detects lane markings;

FIG. 8 shows a flowchart for a method of performing lane detection, inan illustrative embodiment;

FIG. 9 shows a schematic diagram illustrating operation of a neuralnetwork to determining lane markings, in a first embodiment;

FIG. 10 shows a schematic diagram illustrating operation of the neuralnetwork, in a second embodiment;

FIG. 11 show a schematic diagram illustrating operation of the neuralnetwork, in a third embodiment;

FIG. 12 illustrates a road segmentation operation performed at thedetection enhancement module;

FIG. 13 shows a flowchart for a method of extracting trajectories ofnearby vehicles for use in lane detection enhancement;

FIG. 14 shows a flowchart of a method for determining a trajectory of anearby vehicle or road agent from an image or video;

FIGS. 15A and 15B illustrate the effect of using the informationextraction module to improve operation of the lane detection module; and

FIGS. 16A and 16B illustrate the effects of road segmentation data onimproving lane detection results.

DETAILED DESCRIPTION

The following description is merely exemplary in nature and is notintended to limit the present disclosure, its application or uses. Itshould be understood that throughout the drawings, correspondingreference numerals indicate like or corresponding parts and features. Asused herein, the term module refers to processing circuitry that mayinclude an application specific integrated circuit (ASIC), an electroniccircuit, a processor (shared, dedicated, or group) and memory thatexecutes one or more software or firmware programs, a combinationallogic circuit, and/or other suitable components that provide thedescribed functionality.

In accordance with an exemplary embodiment, FIG. 1 shows a vehicle 10.In various embodiments, the vehicle 10 can be a non-autonomous vehicleor an autonomous vehicle. An autonomous vehicle can be a so-called LevelFour or Level Five automation system. A Level Four system indicates“high automation,” referring to the driving mode-specific performance byan automated driving system of all aspects of the dynamic driving task,even if a human driver does not respond appropriately to a request tointervene. A Level Five system indicates “full automation,” referring tothe full-time performance by an automated driving system of all aspectsof the dynamic driving task under all roadway and environmentalconditions that can be managed by a human driver. It is to be understoodthat the system and methods disclosed herein can also be used with anautonomous vehicle operating at any of Levels One through Five.

The autonomous vehicle generally includes at least a navigation system20, a propulsion system 22, a transmission system 24, a steering system26, a brake system 28, a sensor system 30, an actuator system 32, and acontroller 34. The navigation system 20 determines a road-level routeplan for automated driving of the autonomous vehicle. The propulsionsystem 22 provides power for creating a motive force for the autonomousvehicle and can, in various embodiments, include an internal combustionengine, an electric machine such as a traction motor, and/or a fuel cellpropulsion system. The transmission system 24 is configured to transmitpower from the propulsion system 22 to two or more wheels 16 of theautonomous vehicle according to selectable speed ratios. The steeringsystem 26 influences a position of the two or more wheels 16. Whiledepicted as including a steering wheel 27 for illustrative purposes, insome embodiments contemplated within the scope of the presentdisclosure, the steering system 26 may not include a steering wheel 27.The brake system 28 is configured to provide braking torque to the twoor more wheels 16.

The sensor system 30 includes a camera 40 that captures an image of anarea surrounding the environment, such as a roadway in front of theautonomous vehicle. The camera 40 can be a digital camera having anarray of photodetectors for capturing an image. Each photodetector canrecord an intensity of red, green, or blue light in an area of theimage. In other embodiments, the camera 40 can be a video camera thatcaptures video of the roadway while the autonomous vehicle is travellingover the roadway, using the array of photodetectors. The sensor system30 can include additional sensors, such as radar, Lidar, etc.

The controller 34 builds a trajectory for the autonomous vehicle basedon the output of sensor system 30. The controller 34 can provide thetrajectory to the actuator system 32 to control the propulsion system22, transmission system 24, steering system 26, and/or brake system 28in order to navigate the autonomous vehicle with respect to the variousroad agents, wherein a road agent can be a road vehicle other than thevehicle 10 and is generally a nearby vehicle.

The controller 34 includes a processor 36 and a computer readablestorage device or computer readable storage medium 38. The computerreadable storage medium 38 includes programs or instructions 39 that,when executed by the processor 36, operate the autonomous vehicle basedon sensor system outputs. In various embodiments, the processor 36 usesextra road information in combination with lane detection methods toimprove the ability of the lane detection methods in locating the lanemarkings.

FIG. 2 shows a schematic diagram of a lane detection system 200 foridentifying lane markers under several conditions. The lane detectionsystem 200 includes a perception, planning and control module 202suitable for detecting lane markers under favorable or goodenvironmental conditions (e.g., good lighting, no rain or snow, etc.),planning a suitable trajectory based on the lane markers and controllingoperation of the vehicle 10 and a detection enhancement module 204 thatprovides additional information that can be used for detecting lanemarkers when environmental conditions are unfavorable. A sensor, such asthe camera 40, captures an image of the environment 206. The image canbe a plurality of temporally spaced images such as in the form of avideo file.

The perception, planning and control module 202 includes a perceptionsystem 208, a path planning module 210 and a control module 212. Theimage is sent from the camera 40 to the perception system 208. Theperception system 208 determines the location of lane markings from theimage. The perception system 208 can determine the colors of the pixelsof the image and segment the image using pixel colors and determinedfrom the shape of the separated pixel section, the location of the lanemarkings. The lane markings can be determined based on the colorcharacteristics (e.g., red-green-blue (RGB)) of the pixels of the lanemarkings vs. the pixels of the surroundings in the image. In alternativeembodiments, a neural network and/or machine learning algorithm can beused to determine the lane markings. The lane marking can be sent to thepath planning module 210. The path planning module 210 generates a scenefor display and shows the lane markings within the scene. The pathplanning module 210 can also plan a path through the environment basedon the scene and the location of the lane markings. The path is sent toa control module 212 which determines instructions for operating thevarious actuators of the actuator system 32 to follow the path. Theactuators can include, for example, steering, brakes, acceleration, etc.

The detection enhancement module 204 performs various algorithms forextracting road information unrelated to determining the lane markingsvia the color characteristics. For example, the extracted roadinformation can be a road feature, including its shape and contour, aroad segmentation of the image as well as the trajectories of nearbyvehicles and/or nearby objects. Road segmentation refers to identifyingpixels of the image that are road-related vs. those pixels that are notroad-related. The information output by the detection enhancement module204 can be used at the perception, planning and control module 202 toenhance the operation of the lane detection module. The detectionenhancement module 204 can be invoked when the perception, planning andcontrol module 202 determines that there is a lane loss situation inwhich it is unable to identify the lane markers. The detectionenhancement module 204 can then perform enhanced lane detection and sendits results back to the perception, planning and control module 202.

The detection enhancement module 204 includes a cyclic buffer 218 andoperates a lane misdetection mitigation algorithm 216. The cyclic buffer218 can be used to store the temporally spaced images and deliver themto the lane misdetection mitigation algorithm 216 in a usable order. Thelane misdetection mitigation algorithm 216 extracts information from theimage that is not directly related to the lane markings. Suchinformation can include the location of the sides of the road or a shapeof the road within the image. This information can be combined with thedetections from the perception system 208 and provided to the pathplanning module 210. The path planning module 210 can plan a trajectorybased on the lane markers generated using the extracted information. Thegenerated lane markers can also be provided to a monitor 220. Themonitor 220 can display the generated lane markers so that the driverknows what the vehicle is using to plan its trajectory and also toassist the driver if he or she takes over driving from the vehicle. Themonitor 220 can be a head-up display, in various embodiments.

FIG. 3 shows a schematic representation 300 of the components of thelane detection system 200 operable at the processor 36 of the vehicle 10to detect lane markings under difficult environmental conditions. Thelane detection system 200 includes the camera 40, the processor 36, andthe monitor 220. The image is sent from the camera 40 to the processor36. The processor 36 operates one or more of the perception, planningand control module 202, the detection enhancement module 204, aswitching module 302 and a comparison/arbitration module 304.

The switching module 302 sends the image to one or both of theperception, planning and control module 202 and the detectionenhancement module 204. Under a favorable environmental condition, theswitching module 302 directs the image to only the perception, planningand control module 202. Under less favorable conditions, the switchingmodule can send the image to both the perception, planning and controlmodule 202 and the detection enhancement module 204. Under theunfavorable conditions, a first result and a second result can beoutput. The first result is output by the perception, planning andcontrol module 202 while the second result is output by a combination ofthe perception, planning and control module 202 and the detectionenhancement module 204. Alternatively, the detection enhancement module204 can generate the enhanced lane markers on its own.

The comparison/arbitration module 304 generates a final result based onthe first result and the second result, as discussed herein.Additionally, the comparison/arbitration module 304 can output aconfidence value of the final result. In one embodiment, thecomparison/arbitration module 304 compares the first result and thesecond result and generates a warning signal when a difference betweenthe first result and the second result is greater than a selectedcriterion. In another embodiment, the comparison/arbitration moduleselects between the first lane detection result and the section lanedetection result based on a first confidence value generated for thefirst lane detection result and a second confidence value generated forthe second lane detection result.

The monitor 220 displays the final lane detection result output from theprocessor 36 to be viewed by a driver of the autonomous vehicle. Thedriver can use the final lane detection result output when takingcontrol of the vehicle. The monitor 220 can also display otherinformation, such as the warning signal generated at thecomparison/arbitration module 304, confidence values, etc.

FIG. 4 shows a schematic diagram 400 illustrating a first method fordetecting lane markings. Image 402 is sent to both a lane detectionalgorithm 404 and an information extraction algorithm 406. Theinformation extraction algorithm 406 performs road segmentation on theimage 402 and locates objects in the image, such as nearby vehicles andtheir trajectories, and objects on the side of the road. The informationfrom the information extraction algorithm 406 is sent to the lanedetection algorithm 404. The lane detection algorithm 404 uses theextracted information to output lane detection results 408 including aset of marker points that are based on image-based lane detectionmethods using color characteristics of pixels in the image as well asthe information obtained from the information extraction algorithm 406.This additional information can be used to determine or confirm alocation for the lane markings within the roadway.

FIG. 5 shows a schematic diagram 500 illustrating a second method fordetecting lane markings. Image 402 is sent to the switching module 302.The switching module 302 selects whether to send the image along a firstbranch 502 or along a second branch 504. Along the first branch 502, theimage 402 is processed using only the lane detection algorithm 404 (withno additional inputs) to generate first lane detection results 506including a first set of marker points. Along the second branch 504, theimage 402 is processed using the lane detection algorithm 404 withadditional input generated at the information extraction algorithm 406to generate lane detection results 508 including a second set of markerpoints, as described herein with respect to FIG. 4 . Under goodenvironmental conditions, the switching module 302 can select the firstbranch 502. When the lane detection results show signs of misdetectionor when the environmental conditions are deteriorated, the switchingmodule 302 will switch to sending the image along the second branch 504.A switching condition for selecting the first branch 502 or the secondbranch 504 can be based on the image resolution or other parameter, aswell as a review of the lane detection results using only the lanedetection algorithm 404. The image can be fed into a neural network todetermine the environmental condition (e.g., raining, snowing) anddetermine a switching condition for the environmental condition.

FIG. 6 shows a schematic diagram 600 illustrating a third method bywhich the lane detection system 200 detects lane markings. Image 402 issent along both a first branch 502 and a second branch 504. The firstbranch 502 includes processing the image 402 only at the lane detectionalgorithm 404 to generate first lane detection results 602. The secondbranch 504 includes processing the image 402 via both the lane detectionalgorithm 404 and the information extraction algorithm 406 to generatesecond lane detection results 604. The first lane detection results 602and the second lane detection results 604 are both sent to thecomparison/arbitration module 304 which compares the results. When thefirst lane detection results 602 and the second lane detection results604 differ from each other by more than a selected criterion, a flag orwarning is generated. The flag or warning can be sent to a warningsystem 606, which can generate an alert to the driver indicating lanedetection issues, etc.

FIG. 7 shows a schematic diagram 700 illustrating a fourth method bywhich the lane detection system 200 detects lane markings. Image 402 issent along both a first branch 502 and a second branch 504. The firstbranch 502 includes processing the image 402 only at the lane detectionalgorithm 404 to generate first lane detection results 602 and a firstconfidence value 702 for the first lane detections results. The secondbranch 504 includes processing the image 402 via both the lane detectionalgorithm 404 and the information extraction algorithm 406 to generatesecond lane detection results 604 and a second confidence value 704 forthe second lane detections results. The first lane detection results602, first confidence value 702, second lane detection results 604 andsecond confidence value 704 are sent to the comparison/arbitrationmodule 304.

The confidence value can be generated as output of a neural network thatis used in generating the lane detection. Alternatively, the confidencevalue can be generated from contextual information. For example, analgorithm may be trained to perform better in certain scenarios orenvironmental conditions. The suitability of the algorithm can thereforebe a factor in determining the confidence value of the results. Thesuitability of the algorithm can be determined using map data and/orweather conditions.

If the first lane detection results 602 and the second lane detectionresults 604 do not match, the comparison/arbitration module 304 canoutput a final lane detection result 706 having the highest confidencevalue. If both confidence values are low or are less than a confidencethreshold, a flag or warning can be generated to indicate lane detectionissues.

FIG. 8 shows a flowchart 800 for a method of performing lane detection,in an illustrative embodiment. The method reduces a number ofcomputations by waiting until lane markings misdetection occurs beforeadding computation steps for information extraction. The method may alsoconstantly monitor the environment, check for challenging scenarios andengage the enhanced lane detection once these scenarios detected.

The method starts at box 802. In box 804, the system performance andoperation of the lane detection algorithm is monitored for a selectedwait time T, which can be a calibratable wait time. In box 806, theprocessor determines if there has been a loss of lane markings when onlythe lane detection algorithm is being used. If there is no loss of lanemarkings, the method returns to box 804 for the wait time T. If,however, there is a loss of lane markings, the method proceeds to box808. In box 808, the processor determines whether it can address achallenging scenario, such as snow, rain, etc., that has caused the lossof lane markings. If the processor determines that it cannot address thechallenging scenario using the methods disclosed herein, the methodproceeds to box 810, in which the driver is prompted to take overcontrol of the vehicle. If, however, the processor determines that itcan address the challenging scenario, the method proceeds to box 812. Inbox 812, information is extracted from the image. In box 814, the lanedetection algorithm is performed using the additional inputs from theinformation extracted from the image.

In box 816, the lane marking is displayed to the driver at the monitor220. In box 818, the lane markings are sent to other components of thevehicle that support automated driving of the vehicle. In box 820, theprocessor checks to see if the challenging scenario is still in effect.If the challenging scenario is still in effect, the method proceeds backto box 812. If, however, the challenging scenario is no longer ineffect, the method proceed to box 822. In box 822, the processorswitches back to normal lane detection methods (i.e., withoutinformation extraction). The method the proceeds from box 822 to box804.

FIG. 9 shows a schematic diagram 900 illustrating operation of a neuralnetwork 902 to determining lane markings, in a first embodiment. Theneural network 902 includes a plurality of network layers. In a normalmode of operation, (i.e., using only the lane detection algorithm 404),the neural network 902 receives the image 402 and determines (from theRGB coding of the pixels of the image) the presence and location of lanemarkings. In an enhanced mode of operation, the extracted information904 obtained via the information extraction algorithm 406 is providedinto the neural network 902 alongside the image 402. The extractedinformation 904 is provided along a separate channel in the neuralnetwork. The neural network 902 outputs an image 906 and places the lanemarkings 908 within the image 906 for viewing by the driver.

FIG. 10 shows a schematic diagram 1000 illustrating operation of theneural network 902, in a second embodiment. The image 402 is enteredinto the neural network 902. The extracted information is entered into afeatures extraction neural network 1002. Features can be pulled from aselected layer of the features extraction neural network 1002 andintroduced into a corresponding layer of the neural network 902. Theneural network 902 outputs the image 906 with the lane markings 908.

FIG. 11 show a schematic diagram 1100 illustrating operation of theneural network 902, in a third embodiment. The image 402 is entered intothe neural network 902. Data is pulled from a selected layer of theneural network 902 and introduced into a corresponding layer 1102 of thefeatures extraction neural network 1002. The corresponding layer 1102outputs the extracted information 904. Features can be pulled from asubsequent layer 1104 of the features extraction neural network 1004 andintroduced back into a corresponding layer of the neural network 902.The neural network 902 outputs the image 906 with lane markings 908.

FIG. 12 illustrates a road segmentation operation performed at thedetection enhancement module 204. The image 402 is input to thedetection enhancement module 204, which separates, the image 402 into aroad segmentation data set 1202. A first region 1204 of the roadsegmentation data set 1202 includes the road and its contour. A secondregion 1206 includes non-road pixels.

FIG. 13 shows a flowchart 1300 for a method of extracting trajectoriesof nearby vehicles for use in lane detection enhancement. The image 402(or video) is input to an object detection module 1302 which detectsnearby vehicles and objects in the image or video. An object trackingmodule 1304 then tracks the nearby vehicles and objects through multipleframes of the video. A compensation module 1306 compensates for thetracking of motion of the vehicle 10. The trajectory tracking module1308 generates the trajectories for the nearby vehicle, which can beused to generate enhanced lane markings.

FIG. 14 shows a flowchart 1400 of a method for determining a trajectoryof a nearby vehicle or road agent from an image or video. The methodstarts in box 1402. In box 1404, a nearby vehicle or road agent islocated within and image and coordinates are assigned to it in acoordinate frame of the image. In box 1406, a tracker algorithmdetermines if the nearby vehicle is also present in a previous frame ofthe video. Otherwise, (i.e., if this is a new nearby vehicle), themethod proceed to box 1408. In box 1408, a new detected vehicle isgenerated and its coordinates stored for use with respect to a nextframe in the video.

Returning to box 1406, if the nearby vehicle is already present, themethod proceeds to box 1410. In box 1410, the trajectory coordinates forthe vehicle are retrieved from the previous frame(s) or image(s). In box1412, the trajectory coordinates for the previous image(s) aretransformed to a coordinate frame at a bird's eye view (BEV)perspective. In box 1414, the BEV coordinates are updated (i.e., in,straight line driving the BEV coordinates are moved closer to the hostvehicle to compensate for the speed of the vehicle). In box 1416, theupdated coordinates are transformed back into the coordinates system forthe image. In box 1418, the updated and transformed coordinates and thecoordinates on the current frame are concatenated and drawn onto thecurrent frame as the trajectory coordinates. In box 1420, the trajectorycoordinates are stored for use in the next frame of the video. Themethod then proceeds back to box 1404.

FIGS. 15A and 15B illustrate the effect of using the detectionenhancement module 204 to improve operation of the perception, planningand control module 202. FIG. 15A shows a first image 1500 of a roadwaywith lane markers 1502. The first image 1500 also displays marker points1504 generated using only the perception, planning and control module202. Circle 1506 highlights the marker points 1504. As shown in FIG.15A, the marker points 1504 are not aligned with the lane markers 1502.

FIG. 15B shows a second image 1510 with corrected marker points 1512generated using the perception, planning and control module 202 andadditional road segmentation data provided by the detection enhancementmodule 204. Circle 1514 highlights the lane markers 1502 along theroadway and the corrected marker points 1512 generated using theadditional road segmentation data. The corrected marker points 1512 arealigned with the lane markers 1502 to a greater degrees than the markerpoints 1504 shown in FIG. 15A.

FIGS. 16A and 16B illustrate the effects of road segmentation data onimproving lane detection results. FIG. 16A shows a first image 1600displaying lane detection results obtained without the use of roadsegmentation data. The first image 1600 shows a snow-covering road withmarker points 1602. Circle 1604 highlights a kink in the marker points1602, which can cause problems at the vehicle. FIG. 16B shows a secondimage 1610 with corrected marker points 1612 obtained using roadsegmentation data. Circle 1614 highlights the same region as circle 1604in FIG. 16A but shows the corrected marker points 1612 without the kinkshown in FIG. 16A.

While the above disclosure has been described with reference toexemplary embodiments, it will be understood by those skilled in the artthat various changes may be made and equivalents may be substituted forelements thereof without departing from its scope. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings of the disclosure without departing from the essentialscope thereof. Therefore, it is intended that the present disclosure notbe limited to the particular embodiments disclosed, but will include allembodiments falling within the scope thereof

What is claimed is:
 1. A method of navigating a vehicle, comprising:obtaining, at a camera on the vehicle, an image of a road upon which thevehicle is moving; extracting a feature of the road from the image;performing a lane detection algorithm to detect a set of lane markers inthe road using the image and the feature; and moving the vehicle alongthe road by tracking the set of lane markers.
 2. The method of claim 1,wherein the feature further comprises a road segmentation information.3. The method of claim 2, wherein the feature further comprises atrajectory of a road agent on the road.
 4. The method of claim 1,further comprising displaying at a monitor the set of lane markers to adriver of the vehicle.
 5. The method of claim 1 further comprising atleast one of: (i) sending the image to a lane detection module; and (ii)sending the image to both the lane detection module and an informationextraction module.
 6. The method of claim 5, further comprisingobtaining a first set of marker points by sending the image to the lanedetection module and a second set of marker points by sending the imageto both the lane detection module and the information extraction moduleand selecting a final set of marker points from the first set of markerpoints and the second set of marker points.
 7. The method of claim 6,further comprising determining a first confidence value for the firstset of marker points and a second confidence value for the second set ofmarker points and selecting the final set of marker points using thefirst confidence value and the second confidence value.
 8. A system fornavigating a vehicle, comprising: a camera for obtaining an image of aroad upon which the vehicle is moving; a processor configured to:extract a feature of the road from the image; perform a lane detectionalgorithm to detect a set of lane markers in the road using the imageand the feature; and move the vehicle along the road by tracking the setof lane markers.
 9. The system of claim 8, wherein the feature furthercomprises a road segmentation information.
 10. The system of claim 9,wherein the feature further comprises a trajectory of a road agent onthe road.
 11. The system of claim 8, wherein the processor is furtherconfigured to display the set of lane markers to a driver of the vehicleat a monitor.
 12. The system of claim 8, wherein the processor isfurther configured to perform at least one of: (i) send the image to alane detection module; and (ii) send the image to both the lanedetection module and an information extraction module.
 13. The system ofclaim 12, wherein the processor is further configured to obtain a firstset of marker points by sending the image to the lane detection moduleand a second set of marker points by sending the image to both the lanedetection module and the information extraction module and select afinal set of marker points from the first set of marker points and thesecond set of marker points.
 14. The system of claim 13, wherein theprocessor is further configured to determine a first confidence valuefor the first set of marker points and a second confidence value for thesecond set of marker points and select the final set of marker pointsusing the first confidence value and the second confidence value.
 15. Avehicle, comprising: a camera for obtaining an image of a road uponwhich the vehicle is moving; a processor configured to: extract afeature of the road from the image; perform a lane detection algorithmto detect a set of lane markers in the road using the image and thefeature; and move the vehicle along the road by tracking the set of lanemarkers.
 16. The vehicle of claim 15, wherein the feature furthercomprises at least one of: (i) a road segmentation information; and (ii)a trajectory of a road agent on the road.
 17. The vehicle of claim 15,wherein the processor is further configured to display the set of lanemarkers to a driver of the vehicle at a monitor.
 18. The vehicle ofclaim 15, wherein the processor is further configured to perform atleast one of: (i) sending the image to a lane detection module; and (ii)sending the image to both the lane detection module and an informationextraction module.
 19. The vehicle of claim 18, wherein the processor isfurther configured to obtain a first set of marker points by sending theimage to the lane detection module and a second set of marker points bysending the image to both the lane detection module and the informationextraction module and select a final set of marker points from the firstset of marker points and the second set of marker points.
 20. Thevehicle of claim 19, wherein the processor is further configured todetermine a first confidence value for the first set of marker pointsand a second confidence value for the second set of marker points andselect the final set of marker points using the first confidence valueand the second confidence value.